Did You Get What You Paid For? Rethinking Annotation Cost of Deep Learning Based Computer Aided Detection in Chest Radiographs

As deep networks require large amounts of accurately labeled training data, a strategy to collect sufficiently large and accurate annotations is as important as innovations in recognition methods. This is especially true for building Computer Aided Detection (CAD) systems for chest X-rays where domain expertise of radiologists is required to annotate the presence and location of abnormalities on X-ray images. However, there lacks concrete evidence that provides guidance on how much resource to allocate for data annotation such that the resulting CAD system reaches desired performance. Without this knowledge, practitioners often fall back to the strategy of collecting as much detail as possible on as much data as possible which is cost inefficient. In this work, we investigate how the cost of data annotation ultimately impacts the CAD model performance on classification and segmentation of chest abnormalities in frontal-view X-ray images. We define the cost of annotation with respect to the following three dimensions: quantity, quality and granularity of labels. Throughout this study, we isolate the impact of each dimension on the resulting CAD model performance on detecting 10 chest abnormalities in X-rays. On a large scale training data with over 120K X-ray images with gold-standard annotations, we find that cost-efficient annotations provide great value when collected in large amounts and lead to competitive performance when compared to models trained with only gold-standard annotations. We also find that combining large amounts of cost efficient annotations with only small amounts of expensive labels leads to competitive CAD models at a much lower cost.Comment: MICCAI 2022, Contains Supplemental Materia

ELVIS: Empowering Locality of Vision Language Pre-training with Intra-modal Similarity

Deep learning has shown great potential in assisting radiologists in reading chest X-ray (CXR) images, but its need for expensive annotations for improving performance prevents widespread clinical application. Visual language pre-training (VLP) can alleviate the burden and cost of annotation by leveraging routinely generated reports for radiographs, which exist in large quantities as well as in paired form (imagetext pairs). Additionally, extensions to localization-aware VLPs are being proposed to address the needs of accurate localization of abnormalities for CAD in CXR. However, we find that the formulation proposed by locality-aware VLP literatures actually leads to loss in spatial relationships required for downstream localization tasks. Therefore, we propose Empowering Locality of VLP with Intra-modal Similarity, ELVIS, a VLP aware of intra-modal locality, to better preserve the locality within radiographs or reports, which enhances the ability to comprehend location references in text reports. Our locality-aware VLP method significantly outperforms state-of-the art baselines in multiple segmentation tasks and the MS-CXR phrase grounding task. Qualitatively, ELVIS is able to focus well on regions of interest described in the report text compared to prior approaches, allowing for enhanced interpretability.Comment: Under revie

Enhancing Breast Cancer Risk Prediction by Incorporating Prior Images

Recently, deep learning models have shown the potential to predict breast cancer risk and enable targeted screening strategies, but current models do not consider the change in the breast over time. In this paper, we present a new method, PRIME+, for breast cancer risk prediction that leverages prior mammograms using a transformer decoder, outperforming a state-of-the-art risk prediction method that only uses mammograms from a single time point. We validate our approach on a dataset with 16,113 exams and further demonstrate that it effectively captures patterns of changes from prior mammograms, such as changes in breast density, resulting in improved short-term and long-term breast cancer risk prediction. Experimental results show that our model achieves a statistically significant improvement in performance over the state-of-the-art based model, with a C-index increase from 0.68 to 0.73 (p < 0.05) on held-out test sets

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Faeces as a novel material to estimate lyssavirus prevalence in bat populations

Rabies is caused by infection with a lyssavirus. Bat rabies is of concern for both public health and bat conservation. The current method for lyssavirus prevalence studies in bat populations is by oral swabbing, which is invasive for the bats, dangerous for handlers, time-consuming and expensive. In many situations, such sampling is not feasible, and hence, our understanding of epidemiology of bat rabies is limited. Faeces are usually easy to collect from bat colonies without disturbing the bats and thus could be a practical and feasible material for lyssavirus prevalence studies. To further explore this idea, we performed virological analysis on faecal pellets and oral swabs of seven serotine bats (Eptesicus serotinus) that were positive for European bat 1 lyssavirus in the brain. We also performed immunohistochemical and virological analyses on digestive tract samples of these bats to determine potential sources of lyssavirus in the faeces. We found that lyssavirus detection by RT-qPCR was nearly as sensitive in faecal pellets (6/7 bats positive, 86%) as in oral swabs (7/7 bats positive, 100%). The likely source of lyssavirus in the faeces was virus excreted into the oral cavity from the salivary glands (5/6 bats positive by immunohistochemistry and RT-qPCR) or tongue (3/4 bats positive by immunohistochemistry) and swallowed with saliva. Virus could not be isolated from any of the seven faecal pellets, suggesting the lyssavirus detected in faeces is not infectious. Lyssavirus detection in the majority of faecal pellets of infected bats shows that this novel material should be further explored for lyssavirus prevalence studies in bats

The Herschel-Heterodyne Instrument for the Far-Infrared (HIFI): instrument and pre-launch testing

This paper describes the Heterodyne Instrument for the Far-Infrared (HIFI), to be launched onboard of ESA's Herschel Space Observatory, by 2008. It includes the first results from the instrument level tests. The instrument is designed to be electronically tuneable over a wide and continuous frequency range in the Far Infrared, with velocity resolutions better than 0.1 km/s with a high sensitivity. This will enable detailed investigations of a wide variety of astronomical sources, ranging from solar system objects, star formation regions to nuclei of galaxies. The instrument comprises 5 frequency bands covering 480-1150 GHz with SIS mixers and a sixth dual frequency band, for the 1410-1910 GHz range, with Hot Electron Bolometer Mixers (HEB). The Local Oscillator (LO) subsystem consists of a dedicated Ka-band synthesizer followed by 7 times 2 chains of frequency multipliers, 2 chains for each frequency band. A pair of Auto-Correlators and a pair of Acousto-Optic spectrometers process the two IF signals from the dual-polarization front-ends to provide instantaneous frequency coverage of 4 GHz, with a set of resolutions (140 kHz to 1 MHz), better than < 0.1 km/s. After a successful qualification program, the flight instrument was delivered and entered the testing phase at satellite level. We will also report on the pre-flight test and calibration results together with the expected in-flight performance

Common Limitations of Image Processing Metrics:A Picture Story

While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation. Performance metrics are particularly key for meaningful, objective, and transparent performance assessment and validation of the used automatic algorithms, but relatively little attention has been given to the practical pitfalls when using specific metrics for a given image analysis task. These are typically related to (1) the disregard of inherent metric properties, such as the behaviour in the presence of class imbalance or small target structures, (2) the disregard of inherent data set properties, such as the non-independence of the test cases, and (3) the disregard of the actual biomedical domain interest that the metrics should reflect. This living dynamically document has the purpose to illustrate important limitations of performance metrics commonly applied in the field of image analysis. In this context, it focuses on biomedical image analysis problems that can be phrased as image-level classification, semantic segmentation, instance segmentation, or object detection task. The current version is based on a Delphi process on metrics conducted by an international consortium of image analysis experts from more than 60 institutions worldwide.Comment: This is a dynamic paper on limitations of commonly used metrics. The current version discusses metrics for image-level classification, semantic segmentation, object detection and instance segmentation. For missing use cases, comments or questions, please contact [email protected] or [email protected]. Substantial contributions to this document will be acknowledged with a co-authorshi

Understanding metric-related pitfalls in image analysis validation

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.Comment: Shared first authors: Annika Reinke, Minu D. Tizabi; shared senior authors: Paul F. J\"ager, Lena Maier-Hei

Cerebral microbleeds and stroke risk after ischaemic stroke or transient ischaemic attack: a pooled analysis of individual patient data from cohort studies

BACKGROUND: Cerebral microbleeds are a neuroimaging biomarker of stroke risk. A crucial clinical question is whether cerebral microbleeds indicate patients with recent ischaemic stroke or transient ischaemic attack in whom the rate of future intracranial haemorrhage is likely to exceed that of recurrent ischaemic stroke when treated with antithrombotic drugs. We therefore aimed to establish whether a large burden of cerebral microbleeds or particular anatomical patterns of cerebral microbleeds can identify ischaemic stroke or transient ischaemic attack patients at higher absolute risk of intracranial haemorrhage than ischaemic stroke. METHODS: We did a pooled analysis of individual patient data from cohort studies in adults with recent ischaemic stroke or transient ischaemic attack. Cohorts were eligible for inclusion if they prospectively recruited adult participants with ischaemic stroke or transient ischaemic attack; included at least 50 participants; collected data on stroke events over at least 3 months follow-up; used an appropriate MRI sequence that is sensitive to magnetic susceptibility; and documented the number and anatomical distribution of cerebral microbleeds reliably using consensus criteria and validated scales. Our prespecified primary outcomes were a composite of any symptomatic intracranial haemorrhage or ischaemic stroke, symptomatic intracranial haemorrhage, and symptomatic ischaemic stroke. We registered this study with the PROSPERO international prospective register of systematic reviews, number CRD42016036602. FINDINGS: Between Jan 1, 1996, and Dec 1, 2018, we identified 344 studies. After exclusions for ineligibility or declined requests for inclusion, 20 322 patients from 38 cohorts (over 35 225 patient-years of follow-up; median 1·34 years [IQR 0·19-2·44]) were included in our analyses. The adjusted hazard ratio [aHR] comparing patients with cerebral microbleeds to those without was 1·35 (95% CI 1·20-1·50) for the composite outcome of intracranial haemorrhage and ischaemic stroke; 2·45 (1·82-3·29) for intracranial haemorrhage and 1·23 (1·08-1·40) for ischaemic stroke. The aHR increased with increasing cerebral microbleed burden for intracranial haemorrhage but this effect was less marked for ischaemic stroke (for five or more cerebral microbleeds, aHR 4·55 [95% CI 3·08-6·72] for intracranial haemorrhage vs 1·47 [1·19-1·80] for ischaemic stroke; for ten or more cerebral microbleeds, aHR 5·52 [3·36-9·05] vs 1·43 [1·07-1·91]; and for ≥20 cerebral microbleeds, aHR 8·61 [4·69-15·81] vs 1·86 [1·23-1·82]). However, irrespective of cerebral microbleed anatomical distribution or burden, the rate of ischaemic stroke exceeded that of intracranial haemorrhage (for ten or more cerebral microbleeds, 64 ischaemic strokes [95% CI 48-84] per 1000 patient-years vs 27 intracranial haemorrhages [17-41] per 1000 patient-years; and for ≥20 cerebral microbleeds, 73 ischaemic strokes [46-108] per 1000 patient-years vs 39 intracranial haemorrhages [21-67] per 1000 patient-years). INTERPRETATION: In patients with recent ischaemic stroke or transient ischaemic attack, cerebral microbleeds are associated with a greater relative hazard (aHR) for subsequent intracranial haemorrhage than for ischaemic stroke, but the absolute risk of ischaemic stroke is higher than that of intracranial haemorrhage, regardless of cerebral microbleed presence, antomical distribution, or burden